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Overview 



Project Graduation Really Achieves Dreams (GRAD) is an ambitious education reform initia- 
tive designed to improve academic achievement, high school graduation rates, and rates of col- 
lege attendance for low-income students. It is an unusual reform model in that it intervenes 
throughout an entire "feeder pattern" of elementary and middle schools that send students into 
each Project GRAD high school. This report presents results of MDRC's multiyear evaluation 
of the effects of Project GRAD on student achievement at elementary schools in six feeder pat- 
terns. encompassing a total of 52 schools in four cities: Houston. Texas (the original site): At- 
lanta. Georgia: Columbus. Ohio: and Newark. New Jersey. A companion report examines Pro- 
ject GRAD's effects at the high school level in three urban school districts. 

In elementary schtxtls. Project GRAD implements reading and math curricula, with enhanced pro- 
fessional development for teachers. In addition, each elententary school builds support in the 
community for school improvement and college attendance, implements a classroom manage- 
ment program, provides students with access to needed social services, and receives special sup- 
port front local Project GRAD organizations. At tlte high school level. Project GRAD’s model 
assumes that better-prepared students would come from the feeder schools, would benefit from 
special academic counseling mid summer academic enrichment in high school, and would qualify 
for a scholarship to attend college, which is the "cornerstone" of the Project GRAD reform. 

The key findings of this report are: 

• Scores on stale achievement tests at Project GRAD elementary schools in Houston and At- 
lanta improved in the years following implementation of the initiative. However, in an envi- 
ronment of strong state and local focus on state achievement tests, scores improved by simi- 
lar amounts at comparison schools in these same districts. 

• Project GRAD produced statistically significant positive effects on elementary students' 
scores on national achievement tests in Houston and Newark: that is, while comparison 
schools experienced a decline in scores on these tests. Project GRAD schools saw scores re- 
main constant or increase. 

• In Columbus, the implementation of Project GRAD was initially weaker titan in the other 
sites, and this appears to have lowered test scores — both absolutely and relative to compari- 
son schools — in tlte early years of the initiative. 
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Preface 



In ihe past decade, school districts around tlto country 1 have sought to improve struggling 
urban high schools, where high dropout rates, poor student achievement, and low rates of gradua- 
tion and college-going remain all too prevalent. In a field crowded with reform initiatives. Project 
Graduation Really Achieves Dreams (GRAD) stands out as particularly ambitious, focusing as it 
does on improving conditions for high school students before they even reach high school. 

First implemented in one high school in Houston. Texas, in the early 1990s. Project 
GRAD has evolved into a comprehensive reform model that intervenes throughout an entire 
"feeder pattern” of elementary and middle schools that send students into each Project GRAD 
high school. The developers of Project GRAD understand that high schools inherit struggling 
students, making it essential to improve both elementary and secondary schools in order to in- 
crease the rates of high school graduation, college-going, and college graduation. Project 
GRAD combines curricular reforms in the lower grades and the opportunity to qualify for a col- 
lege scholarship in high school with a classroom management program, access to social ser- 
vices. and efforts to promote parental and community involvement at all grade levels. 

With principal support from the Ford Foundation. MDRC has conducted an evaluation 
of Project GRAD in Houston and several expansion districts. This report focuses on student 
achievement at elementary schools in six feeder patterns, encompassing a total of 52 schools 
across four districts: Houston. Texas: Atlanta. Georgia: Columbus. Ohio: and Newark. New 
Jersey. A companion report offers findings on high school student achievement in Houston. At- 
lanta. and Columbus. 

Taken together, the findings from both reports highlight the challenges that urban 
school districts face in significantly improving the academic performance of high school stu- 
dents. While Project GRAD schools made some significant gains in elementary test scores and 
for students at the flagship high school in Houston, in other areas they did not outpace die com- 
parison schools, many of which were also engaged in local and districtwide reforms. Project 
GRAD USA. die dynamic organization that oversees the initiative, has already begun to refine 
its comprehensive approach to respond to some of the lessons suggested by this unusually rig- 
orous evaluation — namely, the "leaky” nature of many feeder systems, the challenges of bring- 
ing reforms to scale, and the importance of taking action at the classroom level to get academic 
gains. Both tlie national organization and the local Project GRAD sites deserve credit for sub- 
mitting their model to a rigorous comparative study. 

This evaluation of Project GRAD is only one of several studies that MDRC has recently 
conducted of high school and district reform efforts, including Career Academies. First Things 
First, and Talent Development. For more information about MDRC's education research and to 
download all MDRC reports, please visit our Web site at www.mdrc.org. 

Gordon Berlin 
President 
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Executive Summary 



Project Graduation Really Achieves Dreams (GRAD) is an ambitious education reform 
initiative designed to improve academic achievement high school graduation rates, and rates of 
college attendance for low-income students. It is an unusual reform model in that it intervenes 
throughout an entire "feeder pattern" of elementary and middle schools that send students into 
each Project GRAD high school. The initiative recognizes that high schools inherit problems 
that have arisen earlier in the education pipeline, making it essential to improve both elementary 
and secondary schools in order to increase the rates of high school graduation, college-going, 
and college graduation. 

Project GRAD schools at all levels build support in the community for school im- 
provement and college attendance, implement a classroom management program, provide stu- 
dents with access to needed social services, and receive special support from local Project 
GRAD organizations. To help students arrive in middle and high school better prepared aca- 
demically, Project GRAD elementary schools implement specific reading and math curricula, 
with enhanced professional development for teachers. At the high school level. Project GRAD's 
model assumes that better-prepared students would come from the Project GRAD feeder 
schools, would benefit from special academic counseling and summer academic enrichment in 
high school, and would qualify for a scholaiship to attend college, which is the "cornerstone" of 
the Project GRAD reform. 

Given Project GRAD's emphasis on early intervention, understanding the program’s ef- 
fects on elementary student achievement is a key step in evaluating its overall effectiveness. 
This report describes the effects of Project GRAD on student achievement at elementary 
schools in six feeder patterns, encompassing a total of 52 schools across four districts: Houston, 
Texas (the original site): Atlanta. Georgia: Columbus. Ohio: and Newaik. New Jersey. MDRC 
— a nonprofit, nonpartisan research organization — conducted a third-party evaluation to de- 
termine the effects of Project GRAD by comparing the changes in student outcomes at Project 
GRAD schools with changes at similar. non-Project GRAD schools in the same districts. (A 
companion report examines Project GRAD's effects at the high school level.) 1 In general. Pro- 
ject GRAD student outcomes are tracked from the implementation of the first components of 
the model at each site until the 2002-2003 school year. The key findings of this report are: 
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• Scores on state achievement tests al Project GRAD elementary schools in 
Houston and Atlanta improved in the years following implementation of 
the initiative. However, in an environment of strong state and local focus 
on state achievement tests, scores improved by similar amounts at com- 
parison schools in these same districts. 

• Project GRAD produced statistically significant positive effects on ele- 
mentary students' scores on national achievement tests in Houston and 
Newark — that Is. while comparison schools experienced a decline in 
scores on these tests. Project GRAD schools saw scores remain constant 
or increase. 

• In Columbus, the implementation of Project GRAD was initially weaker 
than in the other sites, and this appears to have lowered test scores — 
both absolutely and relative to comparison schools — in the early years 
of the initiative. 

The remainder of the Executive Summary describes the Project GRAD model and how 
it was implemented in the school districts, explains how the evaluation was conducted, and 
summarizes the study's findings and explores their implications. 

What Is Project GRAD and How Was It Implemented? 

Project GRAD is unusual in recognizing the interconnection of educational issues at the 
elementary and secondary levels by working at the level of a feeder pattern — a high school and 
the associated elementary and middle schools that feed into it. Over time. Project GRAD has 
evolved from an effort to increase the rate of college-going among students at one Houston high 
school — by offering college scholarships — into a more comprehensive response to the educa- 
tional problems that students at all levels face in scores of schools. 

A complex, multilayered initiative. Project GRAD includes a set of core components 
for all the schools in a feeder pattern as well as components for the schools at each level, as 
described below. 

Components at Project GRAD Elementary Schools 

During the time covered by this study. Project GRAD had two curricular interventions 
at the elementary school level, as well as the components described below that seek to create an 
environment that is conducive to learning. (Currently. Project GRAD supports whatever reading 
and math curricula that participating districts adopt.) 
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• Reading curriculum: Most Project GRAD sites used Success for All (SFA), 
a nationally recognized reading program that focuses on the key elements of 
reading instruction dunng concentrated instructional time (90 minutes each 
day), with die goal of bringing students to grade-level leading by third grade. 

• Math curriculum: Math Opportunities. Valuable Experiences. Innovative 
Teaching (MOVE IT™ Math) was Project GRAD's recommended math pro- 
gram. It offers elementary school teachers professional development and in- 
structional materials organized around the National Council of Teachers of 
Mathematics (N'CT.Vl) Standards program, involves heavy use of manipula- 
tes to address a wide variety of learning styles, emphasizes daily problem 
solving, and introduces algebra in the early grades. 

• Parental and community involvement: Project GRAD seeks to engage 
parents and the community in the work of the schools, build awareness of the 
opportunity to attend college, and support the learning of students. 

• Social services and academic enrichment: One of two programs — Com- 
munities In Schools (CIS) or the Campus Family Support (CFS) Plan (devel- 
oped by Project GRAD) — bring additional social services, academic activi- 
ties. and volunteers into Project GRAD schools to address issues that stu- 
dents and their families face and to build commitment to academic success. 

• Classroom management: Programs developed by Consistency Manage- 
ment & Cooperative Discipline®* (CMCD) SM are designed to produce orderly 
classrooms focused on learning by promoting student responsibility and self- 
discipline and positive relationships among students, teachers, and other 
adults in the school. 

Components at Project GRAD High Schools 

At the high school level. Project GRAD includes tire three components focused on par- 
ent and community involvement, social services and academic enrichment, and classroom man- 
agement. In addition. Project GRAD high schools offer two components: 

• Project GRAD college scholarships are provided to students who have a 
cumulative 2.5 grade point average, graduate within a four-year time period, 
complete a recommended college preparatory curriculum, and participate in 
two summer institutes. Scholarship amounts and criteria vary slightly by site 
but usually average $1,000 to S 1 .5(H) each year during the four years of col- 
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lege. Each Project GRAD high school has a scholarship coordinator who 
provides counseling, tutoring, and college admission preparation. 

• Summer institutes provide an opportunity for qualifying Project GRAD 
students to experience a college campus-based program taught by college 
faculty and to enhance their academic skills. 

Based on encouraging results in its first Houston feeder pattern. Project GRAD ex- 
panded to other feeder patterns within the district. In 1998. Newark. New Jersey, became the 
first site outside Houston to implement Project GRAD: Columbus. Ohio, and Atlanta Georgia 
followed soon thereafter. Currently. Project GRAD operates in five feeder patterns in Houston 
and in 12 school districts and 21 1 schools in eight states across the country, serving more than 
131,000 students. To manage and support each Project GRAD initiative. local not-for-profit 
organizations were established in Houston and the expansion sites. Expansion within the Hous- 
ton schools and to other school districts stretched the capacity of sonve progrant developers to 
support the model's components and prompted the development of a national organization in 
2(XX) — Project GRAD USA — to sustain implementation efforts and to address implementa- 
tion issues across sites. 

Three important points should be noted about the implementation of Project GRAD in 
elementary schools in the four study sites: 

• Although the implementation process differed across sites, the feeder 
patteras of schools examined in this report generally implemented the 
core Project GRAI) components and followed the approach set forth in 
the model. 

• Local situatioas in the school districts meant that the strength of Project 
GRAD's implementation varied. Houston and Atlanta achieved the 
strongest implementation, followed by Newark and. finally. Columbus. 

• Many of the comparison schools were also participating in reform initia- 
tives. likely lessening the treatment contrast between Project GRAD and 
comparison schools, particularly in Houston and Atlanta. 



How Was the Evaluation Conducted? 

The goal of this evaluation is to understand whether Project GRAD changed the academic 
achievement of children in the elementary schools it serves and. if so. how. The study focuses on 
test scores because they are the focus of policy attention and because other typical measures — 
like absence or expulsion rates — are already very low in Project GRAD elementary schools. The 
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evaluation relied on the tests administered by the school districts, which included state achieve- 
ment tests, national achievement tests, or — in Houston — both types of tests. 

To estimate the program's effect on achievement. MDRC used an approach called 
"comparative interrupted time series analysis." The first step in estimating program impacts 
with this design is to compare the change at Project GRAD schools in a given student outcome 
after the school began implementing Project GRAD with the average outcome during a baseline 
period, before implementation. This estimate represents how student performance changed in 
the presence of Project GRAD but does not. by itself, provide a measure of the effect of Project 
GRAD. The next step is to measure the corresponding change during the same period for simi- 
lar schools not implementing Project GRAD. This measurement provides an estimate of how 
student performance would most likely have changed at the Project GRAD schools in the ab- 
sence of the reform. The difference between these two changes is an estimate of the impact of 
the reform — the effects that can be attributed to Project GRAD. 

Project GRAD is typically implemented over several years as individual components of 
the model are put in place, so these findings reflect the initiative at a specific point in its history 
at each site. Being the first district to implement the model. Houston offers more years of fol- 
low-up data than the expansion sites, which were at an early stage in their operation of Project 
GRAD during the years covered by this report. Finally, student mobility into and out of schools 
is common in urban districts. While tire findings presented here include all students at the Pro- 
ject GRAD and comparison schools, tlve findings are similar when the analysis focuses on 
"nonmobile" students who remained at these schools for multiple years. 



How Did Project GRAD Affect Elementary Student Achievement? 

• In Houston and Atlanta, where Project GRAD implementation was 
strong, student scores on state achievement tests at the Project GRAD 
schools improved. During the same period, similar improvements on 
state tests also occurred at the comparison schools, which implemented 
other district- and school-level reforms (often focused on boosting scores 
on state tests). 

Achievement on die Texas state standards-based tests at Project GRAD Houston elemen- 
tary schools improved substantially during the years following live initiative’s implementation. 
However, comparison schools throughout the district made similar progress on these tests, sug- 
gesting that Project GRAD did not improve these outcomes beyond what would have happened 
without the program. The period from 1993 to 2003 was one of substantial progress in students' 
test scores across low-- performing elementary schools in Houston. For example, over live eight 
available years of follow-up. average test scores for fourth-grade math (as measured by the Texas 
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Learning Index) al the Project GRAD schools in the Jefferson Davis feeder pattern rose from 63 
to 82. while scores at the corresponding comparison schools rose from 61 to 81 . 

• Scores on national achievement tests fell at comparison schools in Hous- 
ton and Newark during the study period. Project GRAD frequently pre- 
vented or lessened a similar deterioration in performance on these tests, 
resulting in significant positive effects on elementary student achieve- 
ment relative to national norms. 

In Houston, findings for the third grade demonstrate the pattern of effects. The Stanford 
Achievement Test (SAT-9) — a test comparing students to test-takers nationally — was first 
administered in Houston in 1998. several years after the initial implementation of Project 
GRAD in the Davis High School feeder pattern. Students’ performance on live SAT-9 at the 
comparison schools used for Houston’s three feeder patterns generally declined, whereas scores 
at the Project GRAD schools in two of the three feeder patterns generally remained relatively 
stable or fell by less than at the comparison schools. The net result is a consistent set of statisti- 
cally significant positive effects on elementary-level SAT-9 achievement in both reading and 
math. For example, the analysis suggests that, in the absence of Project GRAD, third-grade 
SAT-9 math achievement throughout the Davis feeder pattern would have fallen to the 25th 
percentile: with Project GRAD, math achievement reached the 38th percentile. 

In Newark during the six years prior to Project GRAD’s implementation, test scores on 
the SAT-9 steadily declined, reflecting the district’s turmoil. During the first two years of fol- 
low-up. scores at the Project GRAD schools stopped declining and improved substantially rela- 
tive to the earlier trend line, whereas no similar break with prior negative trends occurred at the 
comparison schools. These effects were especially pronounced for several grades and subjects. 
For example, the analysis suggests that, in the first year of implementation, average third-grade 
math achievement at the Project GRAD schools reached the level of the 48th percentile instead 
of the 28th percentile — the level that was predicted, had the model not been implemented. Un- 
fortunately. changes in testing in the Newark district prevented longer-term follow-up. so it is 
not possible to determine whether these positive, statistically significant, and substantial effects 
continued. It is important to note also that the positive effects in Newark began before the 
model's instructional components were even implemented, suggesting that the components re- 
lating to classroom discipline and social supports — by themselves — can have effects on aca- 
demic performance. 

• In Columbus, trends in test scores reflect the site's inconsistent imple- 
mentation of Project GRAI). Overall, there was little sustained im- 
provement in test scores at either the Project GRAD or the comparison 
schools during this early follow-up period. There is some indication that 
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Project GRAI) may have produced small, negative impacts on some 
subjects and grades, most of which dissipated over time. 

As has been found in other studies, difficult early implementation of complicated edu- 
cation reforms can temporarily result in stresses on schools and in unintended short-term effects 
on student outcomes. In some follow-up years and grades, scores at the Project GRAD Colum- 
bus schools appear to have fallen slightly below tike baseline averages, while no similar declines 
from the baseline occurred at the comparison schools. By tlve third year of follow-up. these 
negative impacts had largely disappeared, except for declines in fifth-grade math. 



What Are the Implications of These Findings? 

In general. Project GRAD was able to operate in a variety of contexts that differed in 
terms of prior student achievement, local capacity, existing education reforms, and district staff- 
ing rules. The ambitiousness of the initiative's model and its expansion to sites other titan Hous- 
ton required the creation of a national organization (Project GRAD USA) that developed its 
own technical assistance capacity. The efforts to expand into additional feeder patterns in Hous- 
ton and simultaneously into new cities sometimes stretched the capacity of the developers to 
support tl>e model's reforms. 

At the same time, the local context in each district where Project GRAD was attempted 
l»ad important influences both on the success of the model's implementation and on its effects 
on student achievement. This leads to a few observations: 

• In settings that were already mounting reforms focused on improving 
state test scores — as in Atlanta and Houston — Project GRAD does not 
appear to have generated systematic improvements on state assessments 
that were greater than the improvements at the comparison schools. 

• On the other hand, in both Houston and Newark. Project GRAD did re- 
verse declining trends on national achievement tests. This suggests that 
Project GRAD has the potential to help schools improve — or, at least, 
to avoid deterioration in — the more general academic competencies 
measured by some national achievement tests. 

Although data limitations prevent a full examination of this theme. Project GRAD 
schools with reasonably good implementation appear to have achieved comparable improve- 
ments on state tests as similar local schools, while avoiding declines in scores on national tests. 
Some experts argue that a narrow focus on improving student performance on state standaids- 
based tests can have an unintended deleterious effect on student achievement measured more 
broadly. Project GRAD's positive impact on national test scores may help address that concern. 
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• Except in Houston, the Project GRAD programs were still relatively 
early in their life cycle when the data were collected. Many argue that it 
takes at least five years for education reforms to take hold and show re- 
sults. which highlights the possibility that results in Atlanta and Colum- 
bus might still improve. 

This evaluation represents the experience of only four district sites and six feeder pat- 
terns. The expansion sites of Atlanta. Columbus, and Newark were the first of the new districts 
added to the Project GRAD network, which has since expanded to at least eight additional dis- 
tricts. Project GRAD's implementation process has undergone important revisions — many 
growing out of this early experience — that are not captured in the evaluation. 

• Project GRAI) may be most useful in school districts where existing re- 
form efforts may not yet be providing adequate support to improve ele- 
mentary-level iastruction — districts where the model’s programmatic 
and structural elements may meet important needs. 

In some districts — even in low-performing districts that serve large proportions of 
economically disadvantaged students — ongoing reforms may be producing rising achievement 
scores, even though achievement levels may still be lower than desired. This is particularly 
likely to be the case for performance on state-mandated, standards-based assessments. In such 
settings. Project GRAD may not fill a gap in existing efforts to improve elementary-level in- 
struction in ways that help meet the standards, and the initiative may compete with other re- 
forms for attention and support. Even in these contexts, however. Project GRAD may improve 
(or at least prevent the erosion of) student performance on the more general skills that are not 
necessarily measured by state standards tests. 

The key implication is to focus on districts that have low achievement and high levels 
of disadvantaged and minority students, where Project GRAD’s emphasis on elementary-level 
instruction in reading and math and on classroom management and social service supports 
would represent a value-added difference over and above reforms that are already in place. 
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Chapter 1 

Introduction 



The Policy Context 

In ihc current U.S. economy, a high school diploma and postsecondary education and 
training are all-important steps to success. In past decades, young people without the skills and 
credentials provided by a solid high scliool and postsecondary education could find opportuni- 
ties in the labor market, but this option is becoming increasingly rare. Unfortunately, high 
school graduation rates have remained mostly unchanged over the past three decades, leaving 
the United States ranked as seventeenth in the world. 1 In addition, many high school graduates 
lack key skills needed to succeed in later life. The achievement gap between minority and non- 
minority students is of great concern. By the end of high scliool. African-American and His- 
panic students, on average, have skills in both reading and mathematics that are the same as the 
skills of average white students in eighth grade. : Of adults age 25 to 29. less than 10 percent of 
Hispanics complete four or more years of college study, and only 17 percent of black young 
adults have completed a bachelor’s degree.' Low rates of high school graduation coupled with 
low rates of college enrollment have serious ramifications for individuals and their families and 
for the country’s workforce development. 

High school reform efforts face a central paradox: The problems that plague most urban 
high schools are rooted in achievement difficulties that began much earlier than the students* 
arrival in the ninth grade. Educators, researchers, and policymakers point to weak preparation in 
basic subjects in the early grades, coupled with the lack of effective student discipline strategies 
and the absence of sufficient social service supports (especially at schools with high proportions 
of low-income students) as critical factors causing low rates of high school achievement and 
graduation. For at-risk youth in poor communities, low motivation, a weak sense of efficacy, 
and a belief that college is out of reach can compound these issues. 

Project Graduation Really Achieves Dreams (GRAD) is a relatively new initiative that 
originated in Houston. Texas, in 1993. By 2004. it had expanded into 1 1 additional school dis- 
tricts and generated interest at the district, state, and federal levels. 4 This interest and growth 

‘National Center for Education Statistics ( 1999). 

2 Haycock (2001). 

'Center i*n Educational Policy (2(KM). 

4 Project GRAD’ s expansion has been supptwted by a federal line item that provided $20 million in 2003. 
Ihc ford Foundation has provided over $50 million lo ihc initiative. Each Project GRAD community raises 
local dollars, and school districts contribute by paying a percentage of ihc costs of specific components. The 
combined annual budget for the initiative across 12 sites in 2004 was about $75 million, which includes the 

(continued) 
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reflects the fact that Project GRAD's core features are designed to respond directly to problems 
that plague urban school districts and that it offers a multidimensional approach to school re- 
form. Moreover. Project GRAD has positioned itself as an initiative specifically developed for 
the most troubled seltools in a district. It is an ambitious effort that works above the individual 
school level to focus simultaneously on all the students in kindergarten' through grade 12 in a 
set of elementary and middle schools and an associated high school (that is. all the students in a 
feeder pattern). Specifically. Project GRAD recognizes that high schools inherit problems that 
have arisen earlier in the education pipeline and that it is essential to effect change at all school 
levels (elementary, middle, and high school) in order to build a strong base of community sup- 
port to advocate for change and successfully reform high schools. 



What Is Project GRAD? 

Project GRAD is an education reform initiative designed to improve student outcomes 
and close the academic achievement gap between low-income and minority students and their 
more advantaged counterparts. The mission of Pmjeet GRAD is to ensure a quality public school 
education for all children in economically disadvantaged communities, so that high school gradua- 
tion rates increase and students are prepared to enter and graduate from college. Specifically. Pro- 
ject GRAD has two goals: ( 1 ) to ensure that 80 percent of all entering ninth-graders in its high 
schools graduate and (2) to ensure that 50 percent of Project GRAD high scltool graduates go on 
to college." Toward these ends. Project GRAD focuses on grades kindergarten through 12 (K-12), 
including tlte elementary and middle schools that feed into a high school, and — in addition to a 
college scholarship offer — it combines a number of mutually reinforcing reforms that are de- 
signed to increase reading and math achievement, improve classroom behavior, encourage paren- 
tal involvement and community support for school reform, provide social service supports for stu- 
dents and their families, and reduce dropout rates and increase graduation rates among at-risk high 
school students. Given the scholarship and the follow-up that is provided to students who receive 
it. Project GRAD essentially becomes a K-16 initiative. 

Project GRAD originated in Houston. Texas, in 1993. emerging from an extended ef- 
fort by its founder — James Ketelsen. the CEO of Tenneco — and other business leaders, who 
were attempting to increase the rate of college-going in a local high school serving low-income 
students, by offering scholarships to students who qualified. When this offer did not generate a 
significant increase in high school graduation and college enrollment. Ketelsen expanded the 
effort to include the associated middle and elementary schools that fed into the high school and 

administrative costs of the national coordinating organization. Project GRAD USA. TT*e present evaluation 
docs not include a cost analysis. 

'in some sites. Project GRAD also operates in prc-K classes. 

6 Scc Project GRAD USA (2004a. 2tMHb>. 




combined additional reforms intended to increase the chances that students could take advan- 
tage of the opportunity provided by a college scholarship. Based on encouraging results in the 
first feeder pattern. Project GRAD expanded to other feeder patterns within the Houston district. 
In 1998. Newark. New Jersey, became the first site outside Houston to implement Project 
GRAD. Currently. Project GRAD operates in five feeder patterns in Houston and in 1 2 school 
districts and 21 1 schools in eight states across the country, serving more than 131.000 students.' 

Project GRAD's Approach to Improving Student Outcomes 

As with many reform strategies. Project GRAD's principles were crafted in fairly broad 
terms at the outset and then were refined on the basis of early operational experiences and les- 
sons. What lias remained constant is a central premise that all students can be effective learners, 
regardless of their backgrounds, if appropriate and timely programmatic interventions are in- 
fused in the primary grades and if the appropriate supports are provided at the secondary level. 
Project GRAD is not based on one specific educational philosophy or pedagogical approach: 
rather, it is a collection of program components and complementary operational strategies 
backed by research and/or experiential support. As such, the initial choice of components was 
more pragmatic than theory -driven, compared with other reform approaches, and Project 
GRAD has continued to develop and refine both the program's individual components and its 
general approach. 

Figure 1.1 illustrates the Project GRAD approach. Though the basic structure was ini- 
tially presented as a set of five components, additional elements have been added over time, and 
Project GRAD's founders emphasize the importance of operating the initiative as an integrated 
set of programs and supports — structural components and program components that enhance 
each other and that collectively create a consistent educational experience for Project GRAD 
students and schools. 

The Scope of the Project GRAD Effort 

A defining characteristic of the Project GRAD model is its recognition that many of the 
challenges that undermine students' success in high school and beyond begin at a much earlier 
point in their educational careers. Therefore. Project GRAD breaks from the school-by-school 
approach used by many reforms and instead is implemented throughout a "feeder pattern" — 
that is. throughout a cluster of elementary and middle schools that feed into a particular high 
school where the Project GRAD program and scholarship guarantee are put into place. In so 



At the time of this report. Project GRAD sites include Akron. OH: Atlanta. GA; Brownsville. TX: Cin- 
cinnati. OH: Columbus. OH: Houston. TX: Kcnai Peninsula. AK: Knoxville. TN: Long Island. NY: Lorain. 
OH: Los Angeles. CA: and Newark. NJ. 
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The Project GRAD Evaluation 
Figure 1.1 

Project GRAD’s Approach to Improving School Functioning and Student Outcomes 
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doing. Project GRAD seeks to implement a set of reforms that follow students in a feeder pat- 
tern from the elementary level through the end of high school. Simultaneously. Project GRAD 
offers support for students at all grade levels, even if they have not had the benefit of this initia- 
tive in their early school years. Project GRAD does not require the realignment of staff or new 
staff. It is implemented with die existing student body in the schools and with the schools' cur- 
rent administrators and teachers, in an effort to "work with tlie existing assets."' 

Project GRAD's components, academic support, classroom management social ser- 
vices. parental involvement and other activities throughout the feeder pattern are intended to 
motivate students, to increase their engagement and achievement, to provide important skills 
and tools for teachers, and ultimately to help build community expectations. The Project GRAD 
approach as an improvement strategy for middle and high schools is influenced by the extent to 
which real feeder patterns exist where it operates. Since the majority of its efforts to improve 
academic instruction come directly in the early grades (and mostly at the elementary school 
level), much of its force at the middle and high school levels rests on a growing percentage of 
students' arriving at Project GRAD high schools with better academic preparation. 

Operational Strategies 

As detailed in Box 1.1. Project GRAD employs three operational strategies: ( 1 ) creating 
a school and classroom environment that is conducive to learning. (2) strengthening opportuni- 
ties to leant, and (3) enhancing school-level capacity for program management and implementa- 
tion. The initiative draws on specific program components and on the coordination and quality 
assurance provided by the local and national Project GRAD organizations, described in the next 
section. In short Project GRAD combines a scholarship guarantee and a set of interrelated ini- 
tiatives selected to improve the focus and motivation of high school students with a set of cur- 
ricular. instructional, and behavioral reforms designed to improve the preparation of students 
arriving at Project GRAD high schools. 

Key Components of Project GRAD 

One tenet of the Project GRAD approach is a commitment to work with existing stu- 
dents. teachers, and administrators in tire schools that are part of the feeder pattern selected for 
program implementation. As a result, districts that choose Project GRAD are not expected to 



‘See Project GRAD USA (2004a. 2004b). 
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Box 1.1 



Project GRAD's Operational Strategies 

• Creating a school and classroom emironment that Is conducive to learning. 

Project GRAD is typically introduced in schools (hat have discipline problems, weak 
student engagement, and a low sense of efficacy and pessimism about the possibility’ 
of academic success. Project GRAD feeder patterns are frequently described as con- 
taining the lowest-performing high schools and many of die most troubled elemen- 
tary schools. In response. Project GRAD seeks to address students’ social service 
needs, increase parental involvement, strengthen school-community linkages 
(through Communities In Schools / CIS / or Campus Family Support f CFSJ ), pro- 
mote student self-discipline and engagement (through Consistency Management & 
Cooperative Discipline [CMCD SV J) . and create excitement about the possibility of 
going to college (through a scholarship offer and summer institutes on college cam- 
puses and special Project GRAD activities such as the Walk for Success). The annual 
Walk for Success — which includes visits to the homes of students in the feeder pat- 
tern by school staff and community representatives — is sponsored each year by the 
local Project GRAD office to build awareness of the scholarship offer and the goals 
of the initiative. Both CIS and CMCD 8M are offered at all grade levels. Though only 
high school students who meet Project GRAD’s requirements can participate in tire 
summer institutes and receive a scholarship, die Project GRAD scholarship is also 
publicized in elementary and middle schools, to help improve students’ (and teach- 
ers') motivation and create a sense of possibility for real change. 



• Strengthening opportunities to learn. Low student achievement is often one rea- 
son that a district or group of schools adopts Project GRAD. An early goal of the ini- 
tiative is to strengthen instruction in the lower grades, to Ivelp improve die basic aca- 
demic skills of students and to prepare them for the secondary grades. Key Project 
GRAD components focus on literacy (typically. Success for All / SFAJ ) and mathe- 
matics (preferably. MOVE IT "* Math) in grades I through 6. to provide students 
with the skills they need to succeed in other subjects and in later grades. SFA has 
begun a middle school program dial was pilot-tested in Project GRAD sites, and 
MOVE IT™ Math has expanded to die middle school grades.’ The use of these 
middle school curricular components varies from site to site. The summer institutes 
for high school students (some of which are residential in specific sites) also provide 
an opportunity to address specific skills gaps among students and to enrich die edu- 
cational offerings in tlie Project GRAD schools. 



(continued) 





Box 1.1 (continued) 



• Enhancing school-level capacity lor program management and implementa- 
tion. In each Project GRAD site, a local nonprofit organization is formed to sup- 
port the implementation of the program through the feeder pattern! s) and to coor- 
dinate with the local district. This local organization works with staff at the district 
and school-building level to implement the Project GRAD components effec- 
tively. and it plays a key role in building local support for live initiative and school 
reform efforts, by developing strategic partnerships with other institutions and by 
fundraising. Although the degree to which the local organizations are connected to 
the school district varies, a partnership with the district is an important element in 
the Project GRAD implementation strategy, even though the local organizations 
are viewed largely as independent catalysts supporting improvement in tl>e Project 
GRAD schools. The national Project CUM) organization — Project GRAD USA 
— monitors progress, provides technical assistance, coordinates component ser- 
vice delivery, develops new components and approaches, and provides funding to 
the local not-for-profit organizations. Project GRAD USA also regularly convenes 
the Project GRAD network and provides fundraising support. 



When districts do not implement SFA or MOVE IT™ Math in Project GRAD feeder patterns. 
Project GRAD endeavor. to support or supplement the district’s curriculum choice. For exam- 
ple. support might include reading consultants or a reading manager for each school, along with 
special teacher training and materials. 



select specific teachers or to reassign principals or teachers or other staff to implement the pro- 
grant.'" Though Project GRAD is an evolving reform, its key components include: 

• The Project GRAI) college scholarship. The program’s offer of a college 
scholarship is often referred to as the cornerstone of Project GRAD because 
it most directly represents the ultimate goal: increasing student enrollment 
and success in college. Through this component. Project GRAD seeks to 

‘Project GRAD describes iLs structural components as including the feeder system, the local Project 
GRAD organization and Project GRAD USA. community involvement and collaboration, and working with 
the district's cxisung asscLs (Project GRAD USA. 2(MMb). 
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raise the academic expectations of students in kindergarten through grade 12 
(K-12) by providing a financial incentive and college awareness and prepara- 
tion activities. Scholarship amounts and criteria vary slightly by site but usu- 
ally average $1,000 to SI. 500 each year during the four years of college. Pro- 
ject GRAD scholars are high school youth who sign a contract to document 
their intent to meet the scholarship criteria. 1 " The program places a scholar- 
ship coordinator in each Project GRAD high school to help encourage stu- 
dents to pursue tlve scholarship and college access. The scholarship coordina- 
tors provide counseling, tutoring, and information about eligibility for the 
Project GRAD scholarship and other sources of financial support for college. 
They help students prepare for college entrance examinations, and they 
monitor students' progress in meeting graduation requirements. Scholarship 
coordinators build college awareness and guide scholars through the college 
selection and admission process. 

• Summer institutes. Typically offering four to six hours per day of instruc- 
tion and related activities for four to six weeks, the summer institutes provide 
an opportunity for Project GRAD scholars to experience a college-campus- 
based program taught by college faculty and to enhance their academic skills. 
The institutes vary by site but typically include reading, writing, math, sci- 
ence. and enrichment and as needed remedial activities. (See Box 1.2.) 

• Classroom management. Consistency Management <5 Cooperative Disci - 

pline < CM CD s ") is a classroom management and discipline program that 

seeks to build student support for classroom management, responsibility, and 
self-discipline by promoting cooperative learning and positive working rela- 
tionships among students, teachers, and other adults in tlve school. CMC'D 5 ' / 
moves beyond the traditional concept of student discipline as a set of adult 
responses to a student's negative behavior to offer a comprehensive class- 
room management program that is intended to build a slurred sense of re- 
sponsibility among students and adults in the building. Usually. C.V1CD SM is 
put in place in all grades at the Project GRAD schools." 



‘"Project GRAD's college scholarships arc for students who have a cumulative 2.5 grade point average, 
graduate within a four-year period, complete a recommended college preparatory curriculum, and participate in 
two summer institutes designed to expose them to an enhanced curriculum while introducing them to college. 
Students who receive the scholarship also get follow-up support from Project GRAD while they arc in college, 
expanding Project GRAD's teach to K-16. 

1 'Project GRAD USA now has an agreement with C\1CD SM that allows the national organization to man- 
age this component at each site alter two and a half years of program implementation. 




Box 1.2 



The Summer Institutes 

During Iheir four years of high school. Project GRAD students are expected to attend 
two four-week summer iitstitutes on local college campuses. These sessions are 
planned by the Project GRAD staff in conjunction with the college and usually are 
taught in part by college staff. Project GRAD students get real-life perspectives on 
college as they go back and forth to classes, eat in the cafeteria and have access to 
other amenities that make up the college experience. Some sites pres ide stipends or 
make special arrangements for transportation so tltat students can attend the iitstitutes. 
Some sites also have residential programs whereby tire high school students live in 
tire college dorms during their summer experience. 



• Social services and parental involvement. Communities hi Schools (CIS) is 
a national program that brings additional support (that is. volunteers, social 
services, and academic enrichment and support activities) directly into 
schools. CIS is implemented in all grades at Project GRAD schools. In some 
sites, each school has a full-time CIS staff person. The CIS component seeks 
to enhance social and academic support services available through the school 
and to provide targeted assistance to students who have problems outside 
school that may affect their classroom performance. The range of activities 
may include guidance counseling, community outreach, and family case 
management. Project GRAD views parental involvement as an important 
strategy to help build family aspirations for academic success, meeting 
graduation requirements, and going to college. Project GRAD USA has de- 
veloped the Campus Family Support (CFS) Flan, which contains similar 
elements for school districts that have no CIS program. Coordinated by CIS 
or the CFS staff. Project GRAD conducts an annual Walk for Success, dur- 
ing which students’ homes are visited by the Project GRAD staff, district 
staff (including principals and teachers), and community leaders, who en- 
courage parents to connect with their child's school and who explain Project 
GRAD and the scholarship offer. (See Box 1 .3.) 

• A research- based literacy program. The elementary schools in the feeder 
pattern put in place a research-based literacy program. Most, but not all. Pro- 
ject GRAD sites use Success for All (SFA). a nationally recognized reading 
program that promotes comprehensive restructuring of school resources to 
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Box 1.3 



The Walk for Success 

During a time period when students’ homes are seldom visited by school officials 
unless there is a problem. Project GRAD lias initiated a unique strategy to bridge 
home and school — and ultimately to engage parents, community leaders, and other 
key stakeholders in school reform. 

Project GRAD's annual Walk for Success in the fall is a community outreach effort 
in which volunteer principals, teachers, other school-level stall', district administra- 
tors. representatives from the business community, and key community leaders visit 
the homes of students attending Project GRAD schools. It is not unusual for live vol- 
unteers to also include college and university partners, parents, and older students in 
Project GRAD schools or for the total number of participants to include several thou- 
sand people. Typically, teams of two go door to door visiting the homes of ninth- 
graders on a designated Saturday and share background information about Project 
GRAD, encourage the students and their families to sign the scholarship contact, and 
conduct a "needs survey” to see whether the families requite other assistance. The 
contract specifies the eligibility requirements for the Project GRAD scholarship and 
describes the services and activities that the initiative provides. In Houston, volun- 
teers include Spanish-speaking individuals, and tlie information is provided in both 
English and Spanish. 

Project GRAD volunteers in some sites also visit the families of students in the early 
elementary grades (K-2) and in the sixth grade. Parents of elementary and middle 
school students are encouraged to sign a pledge that helps encourage their children to 
pursue academic achievement and college. Some sites hold Walk for Success rallies 
in central locations and showcase high-profile individuals from a variety of fields 
(noted authors, athletes, political leaders, and so on) who seive as motivational 
speakers and role models for students, encouraging them to prepare for college and to 
be optimistic about their future. 

The Walk for Success helps break down the barriers tliat many parents feel exists be- 
tween schools and home, particularly when parents may liave unhappy memories of 
their own school experiences. In essence. Project GRAD brings the scliool to parents’ 
doorsteps. The Walk for Success is an important constituency-building activity for 
Project GRAD, providing a way for an array of stakeholders to become involved in 
the program and to help support school improvement efforts. 
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provide concentrated instructional time (90 minutes each day) for reading, to 
bring students to grade level in this subject area by tl*e third grade. 

• A research-based math curriculum. The elementary schools in the feeder 
pattern also put in place a research-based math curriculum. Project GRAD's 
preferred math component is Math Opportunities. Valuable Experiences, In- 
novative Teaching ( MOVE IT ™ Math), which is organized around the Na- 
tional Council of Teachers of Mathematics (NCTM) Standards. It is a K-8 
professional development program that advocates math instruction based on 
the use of manipulates to address a wide variety of learning styles. 12 
(MOVE IT™ Math emphasizes daily problem solving and introduces alge- 
bra in the early grades.) 

Intended Effects on Student Outcomes 

The Project GRAD components are hypothesized to improve the school climate and 
classroom instruction as shown in the bottom panel of Figure 1.1. The expectation is that the 
program will result in reduced discipline referrals and suspensions, improved attitudes toward 
school, higher expectations, greater time spent on task, and improved teaching and learning. 
These effects, in turn, are expected to lead to improved student achievement, as indicated by 
higher average achievement test scores and greater numbers of students performing at grade 
level, particularly in the lower grades. These improvements in achievement are also expected to 
reinforce the cycle of improved school climate and instruction. 

The momentum created by improved student achievement in tlte lower grades is then hy- 
pothesized to provide a foundation for increased achievement at the middle and high school lev- 
els. as well as prepare students to perform better in existing courses and create a demand for 
higher-level courses. Over time, this is expected to generate increases in both the offerings and the 
completion of more demanding courses. This better high school preparation is in pan motivated 
by the offer of the Project GRAD scholarships, which are able to offer an immediate reward: the 
chance to earn a scholarship to go to college. Finally, this improved student achievement then pro- 
duces increases in the number of students completing high school, going on to college, and receiv- 
ing a degree. The Project GRAD planners envision that success in one feeder pattern will lead to 
adoption of the initiative in other low-performing feeder patterns, up to the point where Project 
GRAD reaches a critical mass in the district and. as a result, develops "staying power." enabling 
the initiative to play an important role in driving change throughout the district. 



1 ‘Project GRAD USA has licensed the right to serve as the national implementation managerof MOVE 
IT™ Math. 




The Project GRAD Implementation Effort 

ll is critically important to understand the scope and complexity of the implementation 
task that Project GRAD undertakes. As soon as the initiative is announced, program staff begin 
working at once in 8 to 10 schools in a district and with whatever administrative unit or units the 
district has in place to manage the set of schools. The Project GRAD components are typically 
phased in over two or three school years, and several of them (CMCD 5 ". SFA. and MOVE 
IT™ Math) require staff development and training prior to implementation. Most components 
require support personnel (managers, coordinators, and facilitators) to help build tire capacity 
needed for teachers and principals to implement them effectively. SFA and MOVE IT™ Math 
require program materials and special classroom artifacts, and CMCD SV also requires special 
artifacts. Finally, the network of component staff must be coordinated by the local Project 
GRAD office in and across the schools in tire feeder pattern. 

As a national initiative. Project GRAD USA coordinates 12 districts, including mral 
sites in Alaska, and anticipates that several additional districts will join in the near future. Cur- 
rently. 21 1 schools are part of the initiative in a network that includes 21 high schools. 34 mid- 
dle schools, and 149 elementary schools. (Alaska's Kenai Peninsula has 7 schools that contain 
both elementary and secondary grades.) Project GRAD Ohio is a special statewide effort in 
which four of the state's largest districts are implementing the program. 



The Project GRAD Evaluation 

This report grows out of MDRC’s independent third-party evaluation of the Project 
GRAD national expansion sites. It discusses tire model's approach to school capacity-building 
and the specific components that are used in the initiative: summarizes Project GRAD's imple- 
mentation in Houston — the initial site — and in three expansion districts (Atlanta. Columbus, 
and Newark): and presents what has been learned thus far about the impacts of this major 
school reform effort on students' academic performance at the elementary school level." A 
companion report analyzes the early effects on high school outcomes in selected sites." 

Key Questions Addressed in This Report 

The key research questions addressed in this report are: 



"The initial expansion sites were Atlanta. GA; Columbus. OH: Los Angeles. CA: Nashville. TN: and 
Newark. NJ. Los Angeles is not included in this analysis because the district’s data system docs not provide the 
historical information that is required by the research methodology in order to assess program effects. Nash- 
ville Ls no longer a Project GRAD site. 

“See Snipes. Holton. OocHittlc. and Sztcjnbcrg (2006). 
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What was the local context in which Project GRAD was implemented, and 
what challenges were laced in its implementation? 



• What are the impacts, or effects, of Project GRAD in Houston, the program's 
original site, on elementary students’ outcomes, over and above the changes 
that would have been observed in the absence of the intervention? 

• What is the impact of Project GRAD on elementary students' achievement in 
the expansion sites through the first several years of implementation? 

• How do Project GRAD's impacts relate to the quality of program implemen- 
tation and the nature of the context in which implementation occurred? 

The Approach for Assessing Program Effects 

In each site. Project GRAD was implemented in feeder patterns of elementary and mid- 
dle schools that fed into a particular high school. Given this structure, it was not feasible to ran- 
domly assign schools or students to pregram and control groups. Therefore, this analysis is 
based on the application of an interrupted time series (ITS) framework to the available data in 
each site. Described more fully in Chapter 3. this approach rests on three basic comparisons: 

• A comparison of post-program achievement at Project GRAD schools and 
pre-program achievement at those same schools to determine whether tlrere 
were improvements over time 

• A compaiison of baseline and follow-up achievement patterns at a set of 
similar schools from the same district, to determine whether these schools — 
without Project GRAD — also experienced improvements over time 

• A comparison of the changes from baseline achievement patterns at both the 
program schools and the comparison schools, to determine whether the im- 
provements at the Project GRAD schools are greater than at the comparison 
schools 

As a framework for the discussion of Project GRAD’s results, this report covers tliree 
important areas. First, it discusses the context in the school districts where Project GRAD im- 
plemented its reforms. Second, it tracks the implementation process and describes how the ini- 
tiative’s components were put in place, to better understand the degree to which Project GRAD 
was in place in the sites over the time period studied. Finally, it addresses whether Project 
GRAD was able to generate improvement outcomes greater than those achieved by a similar set 
of comparison schools from the same district. 
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The Evaluation as a "Differential” Impact Study 

This report focuses on the elementary school level and examines impacts on student 
achievement in reading and math. It is a differential impact study, meaning that it compares 
changes in student outcomes at Project GRAD's elementary schools with changes in student 
outcomes at comparison schools that may be implementing other reforms or that are subject to 
other school improvement efforts undertaken by the district. 

Project GRAD proves to be attractive to districts that are interested in a reform agenda, 
and — in an era of accountability and empliasis on low-performing schools — it is seldom the 
only approach being utilized. For example, almost every school in most urban districts is doing 
something to improve student achievement, and districts are increasingly playing a direct role in 
instructional change and in improvement efforts at the building level. In some cases, districts 
adopted Project GRAD as one strategy for some schools in a broader, districtwide improvement 
plan. As a result, when Project GRAD schools are compared with similar schools, this typically 
means that the comparison schools are implementing other school-level reform strategies and 
approaches and are subject to districtwide improvement efforts. In the proliferation of compre- 
hensive school reform models — and most recently with the advent of No Child Left Behind 
legislation — low-performing schools that have not been targeted for some type of school im- 
provement are rare. These school-level reform efforts are summarized in Chapter 2. which dis- 
cusses Project GRAD's implementation in each site. 

In addition, during the period of Project GRAD’s implementation, there has been a 
heightened climate of oversight and accountability because of federal and state initiatives. To 
varying degrees across the study sites, specific mandates for required improvement in a range of 
student outcomes, including benchmarks and timetables, were set for all schools in the districts. 
As a result, student performance outcomes were routinely reviewed by the district, principals, 
and model developers and program implemented. Further, over time, districts increasingly be- 
came involved in making curricular and instructional decisions at the elementary school level, 
which affected both Project GRAD and the comparison schools. These district initiatives led 
I’roject GRAD to become son*ewhat more flexible in the choice of curricular components. 
Formal and informal networks also supported efforts of schools implementing other non-GRAD 
reforms, and for some this also included coordination and technical support. Specific strategies 
that focused on improving instruction and teacher development emerged; altltough different 
from Project GRAD in their content, they were similar in their intent. In some instances, dis- 
tricts provided schools with other types of supports that were not available to Project GRAD 
schools, and. conversely. Project GRAD scltools received support from districts over and above 
what their non-Projeet GRAD counterpails received. 

Thus, in this evaluation, schools adopting Project GRAD faced a specific, ambitious, but 
policy-relevant standard: Impacts are defined as making more progress than comparison group 
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schools — which were also making considerable efforts lo improve test scores. To make ihe 
analysis clear and lo document the progress thai did occur, findings are presented in three stages: 

1. Were there improvements at the Project GRAD schools over time (from the 
baseline period into the follow-up period)? 

2. What was occurring at similar comparison schools over the same period? 

3. How do changes from the baseline to the follow-up period at Project GRAD 
schools differ from changes at the comparison schools? 

In summary, in studying an education reform effort like Project GRAD, both the 
strength of its implementation and the other local contextual features affecting similar sclxx>ls 
can influence the impacts. Chapter 2. in discussing the implementation of Project GRAD, pre- 
sents a framework that describes the experiences in Houston and in the three expansion sites as 
illustrations of different combinations of implementation strength and local context. This per- 
spective can help readers to interpret the impact findings presented in the later chapters. 



Topics Addressed in This Report 

The remainder of this report is organized as follows: 

• Chapter 2 provides an overview of the Project GRAD model's evolution in 
Houston, the flagship site. It also describes the implementation experience in 
the expansion sites and presents key lessons that have emerged from that ex- 
perience. 

• Chapter 3 describes the basic approach to the report's impact analysis. 

• Chapter 4 discusses program impacts in Houston's elementary grades, and 
Chapter 5 covers impacts in the expansion sites. 

• Chapter 6 summarizes the findings across the Project GRAD sites and places 
them in the broader context of school reform and research on comprehensive 
school reform. 
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Chapter 2 

The Implementation of Project GRAD 



This chapter provides an overview of the implementation of Project Graduation Really 
Achieves Dreams (GRAD) in Houston — the original site — and in the expansion sites covered 
in this evaluation. In interpreting findings from the evaluation, it is important to understand the 
timing and strength of Project GRAD implementation and the context in which the program was 
operating. This helps set expectations about when impacts might be expected to emerge, and it 
illustrates tire nature of the comparisons with non- Project GRAD schools undertaken in each site. 



The Evolution of the Project GRAD Model: The Houston Story 

Project GRAD Houston developed from a business-supported scholarship offer that be- 
gan at Jefferson Davis High School in Houston. Texas, in 1988. The business-school partner- 
ship provided Davis students with university-based summer institutes, mentoring, tutoring, so- 
cial services, leadership development and summer jobs, and college scholarships. However, 
high school graduation and college enrollment rates remained lower than desired, and antici- 
pated improvements did not occur. James Ketelsen — the former CEO of Tenneco and an ac- 
tive leader in the effort to reform Jefferson Davis High School — concluded that the supports at 
the high school level were offered too late to compensate for weak academic preparation in the 
earlier grades. In collaboration with the Houston Independent School District (HISD). Ketelsen 
sought promising reforms intended to enhance learning at the elementary and middle school 
levels and to enable more students over time to take advantage of the college awareness and 
scholarship opportunities in high school. 

Table 2.1 illustrates tire initiation and expansion of Project GRAD in the Houston 
feeder patterns. The Davis feeder pattern consists of a high school, one middle school, and six 
elementary schools. Project GRAD was pilot-tested in this feeder pattern in the 1993-1994 
school year, beginning with the implementation of the classroom management component. 
Consistency Management & Cooperative Disciplme SM <CMCD SV ).' However, not all tire ele- 
mentaiy schools began implementation that year. 2 Project GRAD implementation began at Jack 
Yates High School in the 1996-1997 school year, with the implementation of MOVE IT™ 
Math. In the 1999-2000 school year, the program began expanding into the schools that were 



'See Chapter 1 for a lull description of ihc Pmjcct GRAD components. 

■'Given that not all elementary schools implemented <MCD“ in the 1993-1994 school year, this analysis 
considers 1994-1995 as the fust year of Project GRAD. 
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The Project GRAD Evaluation 
Table 2.1 



Project GRAD Implementation in Houston. Texas 



Key Components for 
Davis Feeder Pattern 


1993-1994 


1994-1995 


1995-1996 


1996-1997 


Consistency Management 
& Cooperative 
Discipline 8 * 4 -* 


Implemented in elementary 
schools' 




Implemented in middle 
schools 


Implemented in Jefferson 
Davis High School 


MOVE IT™ Math-* 




Implemented in elementary 
schools 


Implemented in middle 
schools 




Success for All-* 






Implemented in elementary 
schools and middle schools 




Communities In Schools-* 




Implemented in middle 
schools and Jefferson Davis 
High School 1 ’ 


Implemented in elementary 
schools 




Key Components for 
Yates Feeder Pattern' 


1996-1997 


1997-1998 


1998-1999 


1999-2000 


Consistency Management 
& Cooperative 
Discipline 8 * 4 —* 




Implemented in elementary 
schools 


Implemented in middle 
schools 




MOVE IT™ Math-* 


Implemented in elementary 
schools 


Implemented in middle 

schools 






Success for All-* 




Implemented in elementary 
schools 






Communities In Schools— * 




Implemented in elementary 

schools 




Implemented in middle 

schools' 



(continued) 






Table 2.1 (continued) 



Key Components for 
Wheatley Feeder Pattern 


1999-2000 


2000-2001 


2001-2002 


2002-2003 


Consistency Management 
& Cooperative 
Discipline 8 "— * 


Implemented in elementary 

schools' 

Implemented in middle 

schools' 






Implemented in Wheatley 
High School 


MOVE IT™ Math-* 


Implemented in elementary 
schools' 

Implemented in middle 

schools 








Success for All—* 


Implemented in elementary 
schools 8 




Implemented in middle 
schools 8 




Communities In Schools-* 


Implemented in elementary 

schools' 

Implemented in middle 
schools and Wheatley High 
School 









SOURCES: Opum (2005): Opuni and Ochoa (2002a). 



NOTES: 

"Ryan Jefferson and Laniar Elementary Schools stalled in 1993-1994. Four other Davis elementary schools began in 1994-1995. 

''Communities In Schools (CIS) in the Davis feeder pattern operated prior to Project GRAD. Funding support from Project GRAD began in 1995- 
1996. 

‘CIS at Yates High School predated Project GRAD. 

d Cullcn Middle School began CIS in 1999-2000. Ryan Middle School began CIS before Project GRAD. 

‘Several elementary and middle schools began Consistency Management & Cooperative Discipline 8 " (CMCD 5 ") in 1999-2000. Other elementary 
and middle schools began CMCD 8 " in later years. 

'Several elementary schools began MOVE IT™ Math in 1999-2000. Other elementary schools began MOVE IT™ Math in 2000-2001. 

'Success for All (SFA) began in several elementary schools in 1999-2000. Other elementary schools began SFA in later years. SFA began in 
middle schools in 2001-2002 and in other middle schools in later years. 

''CIS began in several elementary schools in 1999-2000. Other elementary schools began CIS in later years. 






part of the feeder pattern for Phillis Wheatley High School with CMCD SM , MOVE IT™ Math, 
and Communities In Schools (CIS), the social services and parental involvement component.' 

Project GRAD Houston, the initial site, benefited from having some familiarity with the 
core program components prior to their inclusion in the model and in having geographic prox- 
imity with three key component developers, creating a direct feedback loop regarding imple- 
mentation challenges and needed refinements. CMCD SM was developed at the University of 
Houston, was operated by an independent contractor, and initially was pilot-tested in the Hous- 
ton schools. MOVE IT™ Math — the first curricular component that was put in place at the 
elementary schools that feed into Davis High School — was initially an independent program, 
but Project GRAD became the program operator in response to tl>e development demands of the 
scaling-up process. CIS. largely viewed as a dropout prevention program, was an established 
service provider in the Houston schools and already operated at the first Project GRAD high 
school: coincident with Project GRAD's implementation. CIS was refined and strengthened. In 
contrast. Success for All (SFA) existed as a stand-alone reading program at some schools prior 
to its integration into tlve Project GRAD model and was operated by an independent developer 
with a national infrastructure. 

Project GRAD’s design and early experience were heavily influenced by the particular 
challenges faced and opportunities discovered in Houston. This unusual history influenced how 
the approach evolved, was presented to and perceived by potential expansion sites, and was rep- 
licated and scaled up. The Houston district context also shaped both the program and the organ- 
izational structure that emerged in the subsequent expansion effort. 

The Houston setting in the late 1990s was conducive to Project GRAD's development 
and implementation. The district’s priorities and strategic vision were compatible with those of 
Project GRAD, and the newly developed regional administrative structure and feeder patterns 
for student assignment facilitated its implementation. The district was willing to give the Project 
GRAD implementers significant flexibility in the day-to-day operation of the initiative. Further. 
Project GRAD's administration was eased by the flexibility in staffing and staff development 
days that were possible in a district without strong teacher and principal unions and without col- 
lective baigaining agreements covering these areas. As the district permitted Project GRAD to 
expand from its original feeder pattern to multiple high schools. Project GRAD’s developers 
were able to refine the model and to add and eliminate elements during what was an extended 
pilot-test phase. 4 Thus, when Project GRAD expanded to other districts, its developers were 

'The Project GRAD scholarship for Wheatley High Schtxil began earlier, in 1997-1998. In addition, two 
other Houston feeder patterns (the Reagan and Sam Houston patterns) began implementing Project GRAD later 
but. given the shorter implementation period, arc not included in this analysis. 

When Project GRAD was launched in Houston, it also included or pilot-tested other services that were not 
continued in the later feeder patterns or sites, and the initiative built on existing initiatives in HISD. To reinforce 

(continued) 
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optimistic that the structure of the reform was established, and they were ready to specify its 
architecture for additional sites. 



Organizational Evolution as Project GRAD Expands 

In the late 1990s. reports of the improvements in academic achievement at tlie Project 
GRAD Houston schools attracted tlie attention of national funders and other urban districts seek- 
ing a solution to poor academic performance and low rates of high school graduation and college 
enrollment. These early trends were especially noteworthy because they represented the academic 
performance of students who had received only part of tlie full Project GRAD program — that is, 
they had not been exposed to tlie program in elementary or middle school. Such encouraging 
news coming out of Project GRAD’s schools stimulated interest in the expansion effort. 

As the initiative expanded. Project GRAD continued to rely on the core service provid 
ers that had participated in implementing key elements of the reform in Houston. For SFA. Pro 
ject GRAD's national expansion could be accommodated within an existing national organi/a 
tion; for CMCD SM and MOVE IT™ Math, however, the expansion required rapid developmen 
of new organizational structures and expanded delivery systems in order to work outside Hous 
ton. Both components lacked trainers and other resources needed to operate in several sites at 
once. Furthermore. Project GRAD's reliance on CIS (which is a national association of local 
affiliates) for key services presented a need for local organizational development in sites that did 
not have a local CIS program or redevelopment in sites where the existing CIS program was not 
immediately able to take on the role envisioned by the Project GRAD context. Although CIS 
ltad previously operated in Newark. Atlanta, and Columbus, the program in each of these sites 
lwd to refocus the services provided to Project GRAD schools in order to be consistent with the 
approach used in Houston. 

The move to sites beyond Houston surfaced a range of typical expansion issues: quality 
control of the sendee delivery system of developers and contractors, coordination among and 
across components, the relationship of the original site to the growing expansion network of 
independent not-for-profit organizations, tlie need for implementation monitoring, and estab- 
lishing cooperative agreements (programmatic and financial) with school districts. 5 



the leadership skills of pnncipals. who played a central role in implementing the initiative. Project GRAD sup- 
ported their participation in Icadciship training at the Harvard School of Education. At oive p>int in the early years 
of Project GRAD in Houston, the University of Chicago's middle schixil math program was used at the middle 
schixil level, hut this aspect of Project GRAD was not pul of the expansion program. Project GRAD Housti>n 
also was able to offer summer programs to introduce incoming sixth- and mnth-gradcn> to the new schools and to 
case their transition into middle and high school. 

'For a discussion of the implementation progress and challenges, sec Holland (2005). 
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As ihe expansion sites began to operate and worked with the original Project GRAD 
staff in Houston, an inevitable tension arose between the new sites' desire to adapt Project 
GRAD to local circumstances and the program developers' desire for fidelity to the original 
model. During the early phase of the expansion period, new sites were required to implement 
four of the live Project GRAD components, and some sites were resistant to this approach. Over 
time. Project GRAD has moved away from this requirement. It is important to note this tension 
because those seeking to expand school reforms (and other types of social program innovation) 
often must decide whether they wish to replicate the original vision of the reform in new set- 
tings or adapt its core model to make it more appropriate to specific conditions in new settings. 6 

These issues and the continued growth of the network of sites led to tire creation of a 
new organization. Project GRAD USA. which became operational in 2000. This national or- 
ganization focused on implementation issues across the several Project GRAD districts, and it 
supported the local organizations in spearheading implementation efforts in individual school 
districts. Project GRAD USA faced numerous challenges. As new sites were added, it had to 
develop an infrastructure to support expansion and guide implementation. It had to package 
several of the model's components to be more user-friendly, and it had to monitor the quality of 
the components as they operated in new sites. And when district mandates and the initiative's 
strategies were not aligned or were in confiict. Project GRAD USA had to serve as broker be- 
tween the sites and the school districts. Where unions operated in the districts. Project GRAD 
USA and local Project GRAD had to leam ways of working within the context of union agree- 
ments. Most important, the national organization had to help the local organizations position 
Project GRAD so that teachers and principals saw it as an important strategy to n*eet their goals 
and improve scores on the required state assessments. Finally. Project GRAD USA was created 
after several sites had begun implementation. As this organizational evolution occurred, the ex- 
pansion sites received varying levels of support, depending on when they joined tl>e Project 
GRAD network; Newark and Columbus grappled with these issues early on. while Atlanta — a 
newer site — received more support from Project GRAD USA. 



The Expansion Sites Included in the Analysis 

Project GRAD's selection of expansion districts and schools has generally begun with the 
identification of districts facing serious educational challenges and a specific high school that is 
perceived to be particularly troubled, which will serve as the home for the Project GRAD scltolarx 

6 At times, the expansion sites expressed frustration with Project GRAD's requirements about the use of spe- 
cific preferred components that they saw as unable to fully meet local needs. At varying points, some sites argued 
for the option of local choice for one or more of Project GRAD's central components. 

In fact, eventually Project GRAD moved away from specifying specific curricula for math and reading and 
positioned the initiative as a support mechanism for the district’s chosen cureicula 
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and scholarships. All ihe expansion sites were facing serious academic challenges, and tlie feeders 
that were identified as appropriate for Project GRAD served low-income students, most of whom 
were in minority racial or ethnic groups. Across all the new sites, academic achievement in the 
new Project GRAD feeders was below the overall level of performance in the local district and 
tended to decline relative to state or national norms as students advanced through the grades. Fur- 
ther. in tire sites where achievement was measured for several years prior to tire start of Project 
GRAD, using comparable tests, trends in achievement were either declining or at best stable. 
Thus. Project GRAD succeeded in expanding into districts and feeders that met its criteria and 
were in need of a new education initiative to improve academic outcomes for students. 

The school districts included in this report represent a substantial expansion and adapta- 
tion of the Houston-based Project GRAD model to new feeder patterns of schools in very dif- 
ferent settings within a relatively brief time period. Although each of the districts is largely mi- 
nority. urban, and poor, they differ from Houston in important ways, lending support to the idea 
that Project GRAD can take root in many different settings. For example, contextual differences 
from Houston include how (and by whom) the initiative was first introduced, the presence and 
strength of unions, different state and local accountability systems, and variations on the feeder 
pattern concept — all of which affected the process of implementation. The diversity of Project 
GRAD sites is an important finding in assessing the general applicability of the initiative. 

Overview of the Expansion Sites 

Tables 2.2. 2.3. and 2.4 in the following sections detail the implementation timetables 
for the three expansion sites in this study: Newark, Columbus, and Atlanta. Three key themes 
emerge from the information in the exhibits. First, the sites began implementing Project GRAD 
at different times, so the length of follow-up available for an analysis of student outcomes var- 
ies. Second, the model's components were not implemented in the same order across the sites. 
In each site, the launch of Project GRAD began with an announcement of the scholarships, but. 
beyond that, there were considerable differences in how the program unfolded. Choices were 
driven by local perceptions of most pressing needs and by component developers' capacities to 
take on new sites at specific times. Third, there was some variation across the sites in the spe- 
cific curricular approaches implemented. While Project GRAD USA — particularly during the 
period covered by the evaluation — preferred sites to adopt the SFA reading program and 
MOVE IT™ Math, local pressure and desires sometimes dictated other choices. This variety is 
illustrated by the following site profiles, which present how the initiative began in each district, 
the district's context, the role of various organizations, and the accomplishments as Project 
GRAD was implemented. 
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Newark, New Jersey 

Project GRAD Newark was the first expansion site and began implementation in the 
spring of 1998.' The feeder pattern consisted of seven elementary schools, a middle school, and 
a high school. The first Project GRAD high school was frequently described as the district's 
worst comprehensive high school, with low student and faculty engagement, serious discipline 
issues, an emerging gang problem, and low student achievement and graduation rates.'' 

The Newark Public School District was the subject of a state takeover in 1995. resulting 
from concerns about weak educational and financial management, low student engagement and 
attendance, discipline problems, poor academic achievement, and low graduation rates. Further, 
the district was part of a landmark series of state court cases tliat found that the system of educa- 
tional finance had not provided sufficient financial resources to high-need districts to allow 
them to provide an adequate educational opportunity. Immediately prior to Project GRAD’s 
implementation, each school in Newark was required to select one of five comprehensive 
school reform models as part of the remedy. The SFA reading program was one of these mod- 
els. but Project GRAD was not. /\s a result, some schools had invested considerable time inves- 
tigating the five options and had already expressed interest in a model other than SFA. and most 
were unfamiliar with Project GRAD. 

The state-appointed superintendent strongly embraced Project GRAD as part of a 
broader reform strategy. Further, the initiative was supported by two foundations tliat publicly 
pledged to support the initiative for the first five years. A local not-for-profit board was created, 
which included the state-appointed superintendent and community representatives and several 
representatives of one of the key funders. In the early years of implementation, a former corpo- 
rate executive led the implementation process, followed by the former principal of Malcolm X 
Shaba/./. High School. The early implementation activities in Newark predated the creation of 
Project GRAD USA as a not-for-profit organization and of a comprehensive new-site develop- 
ment or orientation plan."' A former principal from Project GRAD Houston was designated as 
the national implementation director, and her role was to provide guidance and coaching to the 
sites. Newark principals and other district administrators visited Houston to observe how the 
model operated and how the components provided training and staff development, but most of 
the implementation process was locally crafted, and negotiations with the developers and con- 
tractors who operated the Project GRAD components were done independently. During this 



'Teacher training few CMCD 8 **, the classrtxmi management component, began in the spring, and C.\1CD SM 
was introduced into the classrooms in the fall of 1998. 

‘’in 2000. the Newark Public School District and Project GRAD fundcis subsequently selected a second high 
school feeder pattern for further expansion, but this group of schools is not included in the MDRC evaluation. 

"'Subsequently. Project GRAD USA developed a comprehensive handbook and training program for new 
sites: New Site Development Handbook (Project GRAD USA. 2004c). 
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